International Journal of Advances in Engineering & Technology, Sept 2011. 

©IJAET ISSN: 2231-1963 



Analytical Classification of Multimodal Image 
Registration Based on Medical Application 

i n 

Mohammad Reza Keyvanpour , Somayeh Alehojat 
Department of Computer Engineering, Alzahra University, Tehran, Iran 

9 

Department of Computer Engg., Islamic Azad University, Qazvin Branch, Qazvin, Iran 



ABSTARCT 

In the last two decades, computerized image registration has played an important role in medical imaging. One 
of the important aspects of image registration is multimodal image registration, where is used in many medical 
applications such as diagnosis, treatment planning, computer guided surgery. Not specified the relationship 
between the intensity values of corresponding pixels, the difference between images contrast in some areas than 
other areas, mapping the intensity values in an image to multiple intensity value in other images, are 
challenging problems in multimodal image registration. Due to importance of image registration in medical, 
identification this challenges seem necessary. This paper will have a comprehensive analysis on several types of 
multimodal image registration methods and will express its affect on medical images. To reach this goal, each 
method will investigate according to its affect on the field of medical imaging and challenges facing each 
method will evaluate analytically. So that recognition these challenges play an effective role in choosing an 
appropriate registration method. 

KEYWORDS: Image registration, medical image registration, multimodal image registration, information 
theory 

I. introduction 

Image registration is the problem of alignment two or more image of different viewpoint, at different 
times or with different kinds of imaging sensors. Registration is important application in many image 
processing and is used in many medical imaging applications. One of the important aspects of image 
registration is multimodal image registration, so that different sensors are used for imaging of an 
image. In this case, the image registration provide tools for gathering information from various device 
and are created a more detailed views.In recent years, multimodal image registration is one of the 
challenging problems in medical imaging. Due to changes in the rotation and size, differences in 
brightness and images contrast, is difficult for a physician to combine mentally all image information 
carefully. Moreover, the radiotherapy techniques using manual adjustment on the MRI and CT brain 
images may require several hours to be analysis [1, 2], Therefore, an image registration technique is 
required until to transfer all image information to a general information system. 
Essentially, image registration methods are divided into three categories based on landmark, 
segmentation and voxel. Major challenges in multimodal image registration are variety of intensity 
images obtained from different sensors. Since voxel based methods is applied directly on image gray 
values, they are more general. Due to importance of medical images, speed and accuracy of 
registration process should be considered. Accordingly, this paper introduces the medical image 
registration methods and will be introduced types of multimodal image registration, then these 
measure compare with measures such as speed, accuracy, computational complexity. Finally, we were 
trying to evaluate this methods effect in the field of medical imaging. The rest of this paper is 
organized as follows: In section 2, the related work and proposed definitions for image registration 
and multimodal medical image registration is introduced. We describe medical image registration 
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methods in section 3. In section 4, the proposed framework for classification of multimodal methods 
is presented and section 5 evaluates these methods. Section 6 includes the conclusion. 

II. RELATED WORK 

Generally, image registration is the process of image component transformation to a coordinate 
system that from image processing viewpoint, the most interesting and possibly most difficult step is 
to determine the proper transformation that transform these components to normal coordinates [3], A 
system for performing image registration algorithms uses of machine vision, image processing, 
machine learning and artificial intelligence [2, 4]. In recent decades, imaging changes identification in 
remote sensing has been much attention [5, 6]. In radiographic, images automatically compare and 
match and in mammography, cancer cases is easily determined [7, 8]. Image registration can be 
applied in the diagnosis and identification steps, such as face detection, handwriting recognition, 
stereo matching and motion analysis [3,4]. One of the important aspects of image registration is when 
various devise used to imaging of a scene. .Therefore, an image registration technique is required to 
transfer all image information to a general information system. In this case, the goal is to display 
images so that to facilitate diagnostic for physicians to find the desired image information similarities 
and differences [9]. More recently developed fully automated methods essentially revolve around 
entropy [10] and mutual information [11, 12]. In this way we can understand that image registration in 
recent years applied to one of the important areas in image processing. 

III. MEDICAL IMAGE REGISTRATION METHODS 

Image registration is the problem of alignment two or more image of different viewpoint, at different 
times or with different kinds of imaging sensors. Registration is important application in many image 
processing and is used in many medical imaging applications. One of the important aspects of image 
registration is multimodal image registration, so that different sensors are used for imaging of an 
image. In this case, the image registration provide tools for gathering information from various device 
and are created a more detailed views.In recent years, multimodal image registration is one of the 
challenging problems in medical imaging. Image registration is used in analyzing medical images for 
diagnosis, in machine vision for stereo matching, in astrophysics to adjust images with different 
frequencies and many other areas. In medicine, patients often in order to better diagnosis or treatment 
is imaging with multiple radiology sensors. Due to changes in rotation or difference in image contrast, 
is difficult for a physician to combine mentally all image information carefully. Therefore, an image 
registration technique is necessary to transfer all image information to an overall system. 
As shown in Figure (1), image registration is used to gather information from various sensors and 
provide more detailed views. Main methods of image registration are divided into three categories: 
intrinsic, extrinsic and non image. Since intrinsic methods are used mainly for multimodal image 
registration, these methods will review. 

Intrinsic methods are classified into landmark, voxel and segmentation based. 

Landmark extraction and image segmentation in some registration methods is difficult while voxel 
based methods are practical and more general [13]. 

3.1 Landmark based registration 

Landmarks are based on anatomy i.e. clear and visible points, which usually are determined by the 
user interaction or are geometric i.e. local areas such as maximum curvature, corners, etc, so that 
usually are defined in an automated method. In landmark based registration, a set of specific points is 
compared with the first image content. These algorithms use of criteria such as average distance 
between every landmark or distance between landmark with the lowest frequency. 
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Target image (MRI) 




CT Registered On MRI 




Figure (1): a multimodal image registration system 

3.2 Segmentation based registration 

Segmentation based registration is rigid where have been extracted from similar image structures to 
be registered, and they can also be deformable model where an extracted structure from one image is 
elastically deformed to fit the second image. Rigid model based approaches are probably the most 
popular methods currently in clinical use. Their popularity relative to other approaches is due to the 
head-hat method which relies on the segmentation of the skin surface from CT, MR and PET images 
of the head. Another cause is the fast chamfer matching technique for alignment of binary structure by 
means of a distance transform. 



3.3 Voxel based registration 

This method directly is applied on the image gray values and does not require to preprocessing and 
user interaction. There are two distinct methods: decrease the content of gray value image to a series 
of scalars and orientations. Second for all images content, has been used through the registration 
process. Methods using all image content, can be applied to almost every field of medicine with the 
use of any transformation to be used, as shown in figure (2), Since multimodal image registration is 
affected by the intensity and methods based on the intensity are applied of gray values image, these 
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category of methods are used for multimodal image registration. 
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Figure (2): medical registration methods classification 



IV. PROPOSED FRAMEWORK 
REGISTRATION METHODS 



FOR MULTIMODAL IMAGE 



Multimodal image registration is one of challenging issue in the field of medical imaging. Therefore, 
choose the method with minimum error for medical image registration may seem important. In this 
section, various methods of multimodal image registration and the challenges of each method will 
explained. In medicine, patients often in order to better diagnosis or treatment is imaging with 
multiple radiology sensors. Due to changes in rotation or difference in image contrast, is difficult for a 
physician to combine mentally all image information carefully. Therefore, an image registration 
technique is necessary to transfer all image information to an overall system. As shown in figure (3), 
using this classification, a suitable method for multimodal medical image registration can be selected. 
This section includes the proposed framework for classification of multimodal image registration 
methods and applications and challenges of each method in the field of medical imaging will be 
evaluated. 

4.1 Information theory based methods 

In recent decades, Information theory is used to effectively in multimodal image registration. In this 
part, measures of information theory and its applications in medical image registration is expressed. 

4-1-1- Entropy 

Shannon entropy for an image is calculated based on probability distribution of image gray values. 

When different sensors are used for imaging, display the intensity of an area in two images, is 

different. Consequently, the aim is reducing variance than the registration obtained . 

The histogram in entropy based methods contains the combination of gray values in each of the two 

images for all corresponding points. When images are aligned correctly, joint histogram shows exact 

clusters of gray values. 

In order to measure the joint histogram distribution of two images, Shannon entropy is used, its 

formula is shown in equation (1). 

H Vi ' / 2' r «)=-Z/ 7 /„/ 2 faQlogp^ (a,b) (1) 

a=I l (x l ,y l ) ( > 

b=I 2 {T a (x v y x )) (3) 

Ii and I 2 are two images that geometrically marked with T a transformation, So that pixels (xi, y0 in II 
with an intensity, is correspond to the pixel T„(xi, yi) in the I 2 with b intensity . While Pn,i2(a> b), that 
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express a highly probable value pairs in the image l { and I 2 is correspond to the image intensity b. 
with finding T a , is minimized entropy transformation and images registered [14] . 
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Multi modal image registration methods 




Entropy 
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Normalized mutual information 



kull back - Leibler distance 
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Figure (3): classification of multimodal image registration methods 

4-1-2- Mutual Information 

Shannon entropy problem is that lower values can lead to false match. For example, if only one 

element have been within the area of overlap of the two images, is produced a sharp peak in the joint 

distribution, Thus reduced the amount of entropy. Mutual information is one of the automatic image 

registration methods in medical imaging where it offers a measure of dependence between two 

images. 

Equation (4) is mutual information definition so that I (T, I 2 , T„) is mutual information measure that 

aligned with T a transformation. 



7(/ 1 ,/ 2 ,r a )= J rY(/ 1 )+W 2 )-Wp/ 2 ,7;) 



(4) 



H (Ii), H (I 2 ) is based on border probability of intensity values in overlapping area of images. 

4-1-3- Normalized Mutual Information 

Size of parts with overlapping images, to impress the measure of mutual information in two ways: 
First, low overlapping, reduces the number of samples, so that is low statistical power of estimating 
the probability. Second, with increasing misalignment, which usually is associated with reduced 
overlap, the measure of mutual information increases. When total entropy increase marginal entropy 
is connected faster. Thus, a measure of normalized mutual information was provided less sensitive to 
changes that are overlapping [14]. 
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4-1-4-Kull back-Leibler Distance 

This method is based on a priori knowledge of the expected joint intensity distribution estimated from 
aligned training images. One of the key features is the use of the expected joint intensity distribution 
between two pre-aligned, training images as a reference distribution. The goal is to align any two 
images of the same or different acquisitions such that the expected distribution and the observed joint 
intensity distribution are well matched. In other words, the registration algorithm aligns two different 
images based on the expected outcomes. The difference between distributions is measured using the 
Kullback - Leibler distance (KLD). The KLD value tends to zero when the two distributions become 
equal. The registration procedure is an iterative process and is terminated when the KLD value 
becomes sufficiently small [15]. The Kullback - Leibler distance between the two distributions is 
given by equation (5): 

,255 



D(PT | IPref) = £ PTCil, 12) log^ggL (5) 



The idea behind the registration technique is thus, to find a transformation T , acting on the floating 
image, that minimizes the KLD between the joint intensity distribution P m and the reference 
distribution P ref . Or, in formula (6): 

r = argmin T £>(P r IIP ref ) (6) 



4.2 Discrete wavelet 

In this method, firstly multimodal images are decomposed by wavelet transformation. Then calculated 
an energy mapping of detailed images from the subclass and is used genetic algorithm to obtain the 
absolute minimum total distance between the energy maps [16]. 

4.3 Intensity Gradient 

The idea of applying this method is to determine similarity between images based on all images so 
that the image structure can be defined by changes in intensity. In this method, an image intensity 
change can be detected via the image gradient and considered the normalized gradient field, which is 
purely geometric information. Computation gradient is less sensitive and allow deal with noisy image 
[17]. 

4.4 Phase Correlation 

The main challenge in automatic multimodal image registration is inconsistency in the intensity 
values and or contradiction between patterns and missing data between images. A method based on 
local phase dependency is not sensitive to the variation intensity, contrast or noise and provides 
efficient method for providing important characteristic of image. In multimodal image registration, a 
feature extraction method based on local fuzzy correlation measure is described. This feature show the 
behavior of local phase structure at various scales in near sharp image features. 
With a reference image and an input image, algorithm making the mapping of local fuzzy dependency 
for both images and estimation of transformation parameters will do registration using an objective 
function [18]. 

4.5 Learning Based Method 

In learning based methods, Instead of using a universal, but a priori fixed similarity criterion such as 
mutual information, a similarity measure is learned, such that the reference and correctly deformed 
floating images receive high similarity scores. In other words, objective function is to maximize the 
correlation between input and reference images and to achieve the desired results, not preset 
preprocessing images [19, 20]. 
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Multi modal image registration is the task of inferring a spatial transformation T for a reference image 
I r and its corresponding floating image I f . Given a similarity function s that quantifies the 
compatibility of aligned reference-floating image pairs, the optimal transformation of (I r , If) is found 
by maximizing the similarity over all possible transformations, such as equation (7): 

T + = arg max T „ s (I r , I f o T) (7) 
Our goal is to train a similarity function s over a sample of pre-aligned image pairs such that the 
empirical cost of mis-registration. Figure (4) show an overview of a learning-based image registration 
system. 



Training phase 




Learning similarity function 



Test phase 




Maximize similarity function 




Optimal 
transformation 



Figure (4): An Overview of a learning-based image registration system 



V. EVALUATION OF MULTIMODAL 
REGISTRATION VARIOUS METHODS 



MEDICAL IMAGE 



Generally, multi modal image registration methods are divided into three categories based on 
landmark, segmentation and voxel. As was expressed earlier, multimodal registration have more 
publicity on some medical images, Since medical imaging requires two principles accuracy and 
speed, in selecting an appropriate method of multimodal image registration according to these 
principles are important. Table (1) and table (2) evaluate amount of influence each of these methods 
on multimodal medical image registration process. The functional measures that considered in our 
evaluation of multimodal medical image registration are as follows: 

• User Interaction: A multimodal image registration method usually is intensity based. They are 
in general fully automatic without the need for user interaction. 

• Speed: A multimodal image registration method must guarantee high speed. 

• Accuracy: A multimodal image registration approach must provide high accuracy in dealing 
with medical data. 

• Computational complexity: This property express that how many iteration dose the algorithm 
need to find the optimal solution. 

• According to studied the evaluation criteria can be seen that, methods based on voxel are 
effectiveness than other methods. 

VI. CONCLUSION 

Not specified the relationship between the intensity values of corresponding pixels, the difference 
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between images contrast in some areas than other areas, mapping the intensity values in an image to 
multiple intensity value in other images, are challenging problems in multimodal image registration. 
Due to importance of image registration in medical, identification this challenges seem necessary. 
This paper had a comprehensive analysis on several types of multimodal image registration methods 
and expressed its affect on medical images area. To reach this goal, each method was investigated 
according to its affect on the field of medical imaging. Results of several studies, indicate that among 
several existing methods in multimodal image registration, voxel based methods is more important. 
Because of voxel based methods are applied on the image intensity values, are more important. Since, 
the main challenge in multimodal registration is diversity of image intensity obtained from different 
sensor, select the method that can identify the multimodal image registration main requirements 
(speed and accuracy) in the medical field, is the other objectives of this paper. 



Table (1): evaluation of multimodal medical image registration methods 
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Table (2): Multimodal registration methods analysis 
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